Method and apparatus for application awareness in a network

ABSTRACT

A method for enforcing a network policy is described herein. In the method, a network socket event request from an application executing in a first context is intercepted by an agent prior to the request reaching a transport layer in the first context. A context refers to virtualization software, a physical computer, or a combination of virtualization software and physical computer. In response to the interception of the request, the agent requests a decision on whether to allow or deny the network socket event request to be communicated to a security server executing in a second context that is distinct from the first context. The request for a decision includes an identification of the application. The agent then receives from the security server either an allowance or a denial of the network socket event request, the allowance or denial being based at least in part on the identification of the application and a security policy. The agent blocks the network socket event from reaching the transport layer when the denial is received from the security server. In one embodiment, the method is implemented using a machine readable medium embodying software instructions executable by a computer.

BACKGROUND Description of Related Art

Virtualized environments rely on firewall rules to protect networks frommalicious traffic. Such firewall rules make decisions based on networksocket information. Port information that is extracted from packetheaders can be the basis for firewall rules to allow or deny traffic.For example, a firewall can allow or deny HTTP traffic by blocking orallowing traffic on the network port assigned by convention to HTTPtraffic, i.e., port 80. Although this approach is easy to apply, theresulting network firewalling is unreliable. For example, applicationscan open up port 80 for HTTP traffic and allow malicious traffic falselyidentified as HTTP traffic. Alternatively, non-malicious HTTP trafficcan occur through a port other than port 80, and be mistakenly treatedas malicious non-HTTP traffic due to the nonstandard port. Further, eventraffic correctly identified as HTTP traffic can be malicious.

Deep packet inspection (DPI) is an alternative to port blocking.Compared to port blocking, with DPI the packet payload is examined todetermine the protocol and can detect malicious contents using a varietyof techniques. Thus, a DPI-based firewall can identify and blockmalicious traffic with significantly more accuracy than port blocking.

Unfortunately, DPI has various disadvantages. Packet inspection requiressignificant processing resources that can increase network latency.Further, DPI requires a huge database of traffic signatures because ofthe large variation in possible traffic signatures to detect malicioustraffic, and frequent updates of the database. Finally, if networktraffic is encrypted, DPI will fail to extract properties of theencrypted payloads of the network traffic. Even if an SSL proxy decryptsthe packets to resolve this latter disadvantage, such an SSL proxydecreases throughput.

SUMMARY

A method for enforcing a network policy is described herein. In themethod, a network socket event request from an application executing ina first context is intercepted by an agent prior to the request reachinga transport layer in the first context. A context refers tovirtualization software, a physical computer, or a combination ofvirtualization software and physical computer. In response to theinterception of the request, the agent requests a decision on thenetwork socket event request to be communicated to a security serverexecuting in a second context that is distinct from the first context.The request for a decision includes an indication of the identificationof the application. The agent then receives from the security servereither an allowance or a denial of the network socket event request, theallowance or denial being based at least in part on the identificationof the application and a network policy. The agent blocks the networksocket event from reaching the transport layer when the denial isreceived from the security server. In one embodiment, the method isimplemented using a machine readable medium embodying softwareinstructions executable by a computer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a network socket event request that isgenerated within a networking layer stack of a virtual machine.

FIG. 2 is a functional block diagram showing an architecture withmultiple virtual machines among which requests for decisions on networksockets and corresponding decisions are communicated.

FIG. 3 is a bounce diagram in a virtual machine context, showing adecision on a network socket event request based on applicationidentification.

FIG. 4 is a bounce diagram in a non-virtual machine context, showing adecision on a network socket event request based on applicationidentification.

FIG. 5 is a functional block diagram showing an architecture withmultiple virtual machines among which statistics about data flowsthrough requested network sockets are communicated.

FIG. 6 is a bounce diagram in a virtual machine context, showing theaggregation of statistics about data flow with virtual machines.

FIG. 7 is a bounce diagram in a non-virtual machine context, showing theaggregation of statistics about data flow with non-virtual machineagents.

DETAILED DESCRIPTION

FIG. 1 is a block diagram showing virtual machine 20 in the process ofissuing a network socket event request that is generated within anetworking layer stack of a virtual machine. A virtual machine 20 ismanaged by virtualization software 30 a executing on physical server 10a. Virtualization software 30 a can also manage additional virtualmachines. Virtualization software 30 a can be native or hosted, andmanages one or more virtual machines, permitting multiple concurrentinstances of operating systems on the same computer hardware resources.The computer hardware resources include physical servers 10 a to 10 z,interconnected by network 2 shown as a network cloud. Physical servers10 a to 10 z include processors, memories, and nontransitory computerreadable media with computer readable instructions executable by acomputer performing the technology described herein. Any number ofservers 10 a-10 z may reside on network 2, and any number of virtualmachines 20 may reside on each physical server 10 a-10 z. For example,virtualization software 30 y is executing on physical server 10 y, andphysical server 10 z does not have executing virtualization software.The servers 10 a-10 z may include a security server.

Within virtual machine 20, multiple networking layers may be stacked,with the physical layer conceptually located on the bottom of the stack.The physical layer is shown as virtual machine hardware 28 a. Above thevirtual machine hardware 28 a is data link layer 27 a. Above data linklayer 27 a is network layer 26 a. Above network layer 26 a is transportlayer 25 a. Above transport layer 25 a isapplication/presentation/session layer 21 a. An application 22 a may beexecuting in application/presentation/session layer 21 a.

Application 22 a sends a network socket event request 23 a. A networksocket is an endpoint with a local address and local port. A resultingnetwork connection includes a source IP, source port, protocol,destination IP, and destination port. Connection-oriented sockets suchas TCP sockets may have a connection state, though not connectionlesssockets such as UDP sockets. Network socket event request 23 a may be astatus change in a network socket, for example requested UDP or TCPevents such as network open, network close, and listen.

A transport layer interface 24 a is positioned between transport layer25 a and application/presentation/session layer 21 a. In one embodiment,transport layer interface 24 a may intercept network socket eventrequest 23 a from application/presentation/session layer 21 a prior tothe network socket event request 23 a reaching transport layer 25 a.

Examples of transport layer interface 24 a are the Transport DriverInterface (TDI) and the Windows Filtering Platform (WFP) on Windowsplatforms. In other embodiments, transport layer interface 24 a isprovided on Linux platforms, Mac OS platforms, or other platforms.

Requested network events can be tied to the requesting application asfollows. In some embodiments, TDI clients above transport layer 25 a,such as afd.sys, communicate using I/O request packets (IRPs) with TDItransports such as TCPIP.sys and TCPIP6.sys. Because the IRP isgenerated in context of application 22 a, transport layer interface 24 acan identify application 22 a as the source of network socket eventsthat start a network connection such as OPEN and LISTEN. For example,transport layer interface 24 a can identify the process ID of requestingapplication 22 a from the IRP, and then map the process ID to the binaryimage of application 22 a to the process ID. During the course of thenetwork connection, application 22 a may generate other network socketevents such as SEND, RECEIVE, and CLOSE. Because TDI clients also useIRPs to generate these events, transport layer interface 24 can identifyand map them to the requesting application 22 a and the process ID inthe same manner.

An alternative to the transport layer interface 24 a is a layeredservice provider that can allow or block the network socket eventrequest 23 a and can reside conceptually above a base transportprovider. For example, a Winsock or Winsock 2 service provider interface(SPI) can be implemented by a module that allows or blocks the networksocket event request 23 a, while relying on an underlying base TCP/IPstack.

Virtual machine 20 may rely on another virtual machine distinct fromvirtual machine 20 (such as security virtual machine 80 shown in FIG. 2)to decide whether to allow or deny the network socket event request 23a. Security virtual machine 80 may base its decision on policies thatmay be centrally-managed. In some embodiments, an agent 29, discussedbelow with reference to FIG. 2, in virtual machine 20 communicates withtransport layer interface 24 a, and sends a request for a decision onwhether to allow or deny a network socket event to security virtualmachine 80. The request may include information about the application 22a such as application file name, application executable hash,application identifier and user/domain of application 22 a. Agent 29receives a decision from security virtual machine 80 and then allows ordenies network socket event request 23 a.

In another embodiment, a network policy might be enforced by a componentdifferent from the security virtual machine. The security virtualmachine consumes the application information from the network socketevent and evaluates its network policy with this applicationinformation. If a match is found such that the application informationidentifies an application subject to the network policy, the securityvirtual machine generates one or more appropriate firewall rules thatare pushed to an enforcement engine. The enforcement engine can resideon its own physical server machine or share the physical server machinewith another part of the described technology.

Based on the decision on whether to allow or deny a network socketevent, agent 29 either forwards network socket event request 23 a totransport layer 25 a or discards it. If agent 29 forwards network socketevent request 23 a to transport layer 25 a, network socket event request23 a is processed on a layer-by-layer basis by transport layer 25 a,network layer 26 a, data link layer 27 a, and virtual machine hardware28, followed by resulting network activity via the network 2.

Physical server 10 z does not have executing virtualization software.Physical server 10 z has application/presentation/session layer 21 z,application 22 z, network socket event request 23 z, transport layerinterface 24 z, transport layer 25 z, network layer 26 z, data linklayer 27 z, and hardware 28 z. Network socket event request 23 zfunctions in a manner similar to network socket event request 23 a, butin a non-virtual context.

FIG. 2 is a block diagram showing an architecture with virtual machine20 including agent 29, security virtual machine 80, other virtualmachines 90, network firewall module 96, network policy managementmodule 98, and application identification module 100. A networkadministrator can determine the network policy via a network policymanagement module 98. Firewall rules implementing the network policy canbe determined by security virtual machine 80. The applicationidentification module 100 can provide application identificationinformation, to assist the security virtual machine 80 in making adetermination on whether to allow or block a network socket eventrequest from agent 29 of virtual machine 20. The enforcement of thisdetermination can be carried out by virtual machine 20 which could blockor allow the network socket from further processing within the virtualmachine 20, or by firewall module 96 which could block or allow thespecific network connection.

Agent 29 in virtual machine 20 may be implemented as a computer programthat runs in the background, as a service or daemon. The various virtualmachines and modules can be on one or more physical servers. In anotherembodiment, security virtual machine 80 can be replaced or complementedby a security module in virtualization software 30 a or a physicalappliance residing on network 2 (shown in FIG. 1). In variousembodiments, the network firewall module 96 can be virtual or physical.

FIG. 3 is a bounce diagram in a virtual machine context, showing adecision on a network socket event request based on applicationidentification. In one embodiment, agent 29 sends a request for adecision on whether to allow or deny a network socket event 81, tosecurity virtual machine 80, via network 2. To make the decisionrequested by virtual machine 20, the security virtual machine 80 relieson application information about application 22 which sent the networksocket event request 23 (from FIG. 1) that prompted the request for adecision on whether to allow or deny a network socket event 81 fromapplication identification module 100. Accordingly, the request 81includes application context information such as application name, ahash of application's executable file, some other applicationidentifier, or the user/domain of the application 22.

The security virtual machine 80 sends request for applicationidentification 101 to application identification module 100, via network2. The application context information is used by the applicationidentification module 100 to generate more information on theapplication 22 which sent the network socket event request 23, such asproduct name, vendor name, application category, application threatlevel, etc. Firewall rules of a network policy on the security virtualmachine 80 can be based on any of this application metadata

The request for application identification 101 leads to a match betweena signature of the application initiating the network socket eventrequest, and a reference application signature in a signature databaserelied on by the application identification module 100.

In various embodiments, the application signature is based on at least afilename of the executable file of the application, a hash of anexecutable file of the application, and/or the executable file of theapplication. The application identification module 100 responds back tothe security virtual machine 80 via network 2 with applicationidentification information 102.

Examples of application identification information 102 are applicationname, version, category, manufacturer, trust level, and threat level.The application identification information 102 can be used by thesecurity virtual machine 80 to implement firewall rule-based decisionsabout whether to allow or deny network socket event requests.

In various embodiments the firewall rules resulting from a networkpolicy, and/or the network policy are stored and updated at the securityvirtual machine 80, or a centralized network policy management moduleseparate from and accessible to security virtual machine 80. Thecentralized network policy management module may be on a separatecentralized network policy management server or share a physical serverwith a virtual machine. Via the centralized network policy managementmodule, an administrator can define a network policy that determines thefirewall rules. New rules may be pushed to the security virtual machine80 (and other security virtual machines) according to a security virtualmachine registration scheme.

-   -   Example network policies are:        -   (i) Block/allow all traffic of protocol X initiated by            application Y when receiving a connection state message Z,            such as ‘Block all TCP traffic initiated by uTorrent when            receiving a SYN_SENT event_.        -   (ii) Block/allow all TCP traffic initiated by application Y.        -   (iii) Block/allow all network traffic initiated by            application Y belonging to category Z (P2P for instance)        -   (iv) Block/allow all network traffic initiated by            applications made by vendor Z.

In some embodiments, the application identification is advantageous,because of the reduction or elimination of deep packet inspection invirtual machine 20 or other virtual machines 90 in connection withapproving or denying network socket event requests, without sacrificingaccuracy in identifying applications that request network socket events.

The application identification module 100 may also be implemented as acloud based application identification service. In other embodiments,the application identification module 100 is located in security virtualmachine 80, in virtualization software 30 a, or in another virtualmachine accessed by network 2. Such relatively centralized embodimentsminimize the overhead in the application signature updates. Theapplication identification module 100 contains a central signaturedatabase that maps application signatures to application identities. Thecentral signature database decreases the number of locations that relyon signature updates. The signature may be a sufficiently completeindication to identify the application requesting the network socketevent. In other embodiments, the indication may be insufficientlycomplete to identify the application, but nevertheless a sufficientlycomplete indication to identify the application as safe (such that thenetwork socket event should be allowed) or unsafe (such that the networksocket event should be denied).

In yet another embodiment, the application identification module 100 islocated in virtual machine 20, although this can have the disadvantageof requiring application signature updates at every virtual machinewhich requires decision on whether to allow or deny network socket eventrequests.

To make the decision on network socket event 82, the security virtualmachine 80 applies a network security policy to the applicationidentification information 102, which results in firewall rulesimplementing the network security policy. The decision on network socketevent 82 is sent back to the virtual machine 20 which sent the requestfor decision on network socket event. In another embodiment, thedecision on network socket event 82 is sent back to a firewall thatenforces the decision. Such a firewall can be a virtual firewall or aphysical firewall. Firewall policies and updates for the securityvirtual machine 80 can be communicated from network 2, and, in someembodiments, from a separate policy management module (not shown).

In yet another embodiment, the security virtual machine 80 alsoprocesses requests for decisions on whether to allow or deny networksocket events, for other virtual machines connected via network 2. Othervirtual machines send requests for such decisions to the securityvirtual machine 80. To make the decisions requested by other virtualmachines, the security virtual machine 80 relies on applicationinformation that is requested from application identification module100, which provides the security virtual machine 80 with applicationidentification information. The security virtual machine 80 appliesfirewall policies to the application identification information, andsends the resulting decisions on network socket events back to thecorresponding other virtual machines 90 which sent the requests fordecisions on network socket events.

FIG. 4 is a bounce diagram in a non-virtual machine context, showing adecision on a network socket event request based on applicationidentification. The operations are similar to FIG. 3. However, aphysical security server replaces the security virtual machine, and anon-virtual machine agent replaces the virtual machine agent. In anotherembodiment, the physical security server also processes requests forwhether to allow or deny network socket events for other non-virtualmachine agents.

Other embodiments combine aspects of FIGS. 3-4. For example, non-virtualmachine agents and virtual agents can be combined. Non-VM agents can beused with a security virtual machine. VM agents can be used with aphysical security server.

FIG. 5 is a block diagram showing an architecture with virtual machine20, security virtual machine 80, other virtual machines 90, and dataflow visibility module 110. The various virtual machines and modules canbe on or more physical servers. Within virtual machines, the TDI/WFPtransport layer interface can collect statistics on network sockets.Outside the virtual machines, any module gathering or trackingconnection-level statistics can perform the same. Security virtualmachine 80 can collect statistics on data flows through networkconnections requested by virtual machine 20 and other machine 90. Dataflow visibility module 110 can request and receive the aggregatedstatistics. In another embodiment, security virtual machine 80 can bereplaced or complemented by a security module in virtualization software30 a or an appliance residing on network 2 (shown in FIG. 1). In otherembodiments, the network firewall or host connection tracking module cancollect statistics on data flows through network connections requestedby virtual machine 20 and other machine 90 (for example, informationabout applications on a per-connection basis).

FIG. 6 is a bounce diagram in a virtual machine context, showing theaggregation of statistics about data flow with virtual machines. In oneembodiment, virtual machine 20 sends statistics about data flow 83(through the requested network sockets of virtual machine 20) tosecurity virtual machine 80, via network 2. Other virtual machines 90also send statistics about data flow 95 (through the requested networksockets of their respective virtual machines) to security virtualmachine 80 via network 2. Such statistics can be sent to the securityvirtual machine 80 at intervals, e.g. every 30 seconds. The networksockets can be requested and approved as discussed in connection withFIGS. 3-4.

Security virtual machine 80 aggregates the statistics about data flow 83from virtual machine 20 and the statistics about data flow 95 from theother virtual machines 90. The aggregated statistics can be processed toindicate network flow information as bytes/packets per application, peruser, per virtual machine, etc. In some embodiments, aggregatedstatistics per application are particularly reliable, because of theapplication identification process discussed in connection with FIGS. 1and 2. In turn, such aggregated statistics can be considered inmodifying firewall policies for subsequent decisions on requests fordecisions on network socket events. Data flow statistics through networksockets that are approved under such modified firewall policies can beaggregated as shown. Data flow visibility module 110 requests statisticsabout data flow 112. The security virtual machine 80 responds with theaggregated statistics 111.

FIG. 7 is a bounce diagram in a non-virtual machine context, showing theaggregation of statistics about data flow with non-virtual machineagents. The operations are similar to FIG. 8. However, a physicalsecurity server replaces the security virtual machine, and a non-virtualmachine agent replaces the virtual machine agent, and other non-VMagents replace other virtual machines.

Other embodiments combine aspects of FIGS. 7-8. For example, non-virtualmachine agents and virtual agents can be combined. Non-VM agents can beused with a security virtual machine. VM agents can be used with aphysical security server.

Examples of architectures that can implement the disclosed technologiesare hypervisor and other virtualization products by Citrix, Microsoft,VMWare, and the Xen community.

While the present invention is disclosed by reference to the preferredembodiments and examples detailed above, it is to be understood thatthese examples are intended in an illustrative rather than in a limitingsense. It is contemplated that modifications and combinations willreadily occur to those skilled in the art, which modifications andcombinations will be within the spirit of the invention and the scope ofthe following claims. What is claimed is:

1. A method for enforcing a network policy on an application executingwithin a first context, the method comprising: intercepting, by an agentexecuting in the first context, a network socket event request from theapplication before the network socket event request reaches a transportlayer in a network stack of the first context; sending, by the agent toa security server executing in a second context, a request for adecision on whether to allow or deny the intercepted network socketevent, the request for the decision including an application identifier;receiving, by the agent, the decision from the security server, thedecision being an allowance or a denial of the network socket eventrequest, the decision being based at least in part on the indication ofthe identity of the application and a network policy; and preventing, bythe agent, the network socket request from reaching the transport layerin the first context when the decision is the denial of the networksocket event request.